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SYSTEM AND METHOD FOR PROCESSING used to acquire the best compression effect. First, we explain 

A COMPRESSED DATA STREAM the intra-frame compression. Video data is composed of a 

series of frames. To provide an effect of persistence of 
vision, thirty frames are typically shown each second. The 

FIELD OF INVENTION 5 MPEG video compression includes steps of color plane 
Hie present invention relates to a system and method for ""Pj"*. discrete cosine transform (DCI), numerical 

processing a compressed data stream. j^? g ' Zlg_Za8, d y namic l * n & h compression encod- 

DESCRIPTION OF THE PRIOR ART Please refer to FIG. 1(B). In a typical computer graphical 

, , • . *» . . , . t . 10 environment, video data of a frame are recorded as arrays of 

l»£ ™ 7 ^ technologies pixels> Md each ^ ^ be ^ J 

have experrenced amazing advancement. n,ese technolo- or blue (RGB). Scientists have found mat humaf eyes 

pes have benefited not only traditional computers, but abo m less t0 ch , fa ho ^ ^ 

Z 5? , houj f h °W, h a PP hances ; s « ch , « VCD peopleperceiveUttleorToefFectofvideodatalostduetothe 

players, MP3 players, etc. With the new technology, more 15 luminance attribute o£ the ^ ^ memo( , 

and more functions are added and greater convenience is mvolves , ransferrin RGB to YUV data where Y is a 

provided as people use these electronic appliances. luminance plane and U and V are chrominance planes. 

One of the most noticeable changes introduced by recent Taking 16*16 pixels as a macro block, the transformation 

computer technology is the inclusion of images, video, and radio of RBG to YUVplanes is typically 4:1:1 (step 121). By 

audio to enhance the previous text-only user interfaces. In 20 using the color plane sampling, the amount of data is 

the age of multimedia, the amount of information to be reduced 

iDCrcaseS / f eatl5 :' p , articular ^ m Wlfcafon. Please refer to FIG. 1(B). Next, for each YUV block that 

as digital movies and remote learning. Therefore .processing has8x8 pixeIs , discrete cosine transformation (step 122) is 

very large amounts .of information is a key problem to solve p erf orme d. After the transformation, the data of each block 

in many new multimedia applications. 25 £ quantized by mcKat criterion of ' quantization J£££ 

One popular method of handling laige data files is to to different needs (step 123) 

compress the data for storage or transmission. Different After the transformation and the quantizing of each block, 

kmds of data are typically compressed using different meth- the two dimension data is scanned into one dimension data 

rwT^rT mP T°i n methods include m , ^ direction „ ^ owo fa nG 1(c) Ne one 

Sh? • ' i h ™ o ^l 60 , • ^ «^° n P 6 *™ 5 a d y nunic ^ compression encoding, such as 

methods include MP3 RealAudio, MPEG and AVI. For Huffman encoding, to the one dimension array (step 124). 

digital data and executable instructions, common compres- „ . . . 4 c . . . 

sion methods include ZIP, ARJ, and TAR. These methods of ^LE^ """^T^ m f° h ° ^T' 

compression can be classified into two categories, lossy ^ f° m P ress,on method * tains a "'g^r 

compression and non-lossy compression. In the former 35 £ T^™, ' T * mcorporatm 8. "tter-frame compres- 

category, natural characteristics of the compressed data are "^2?. * T ^ 

found and a level of information accuracy of data is sacri- ^ de odaU streams in neighbonng frames have sUongre^m- 

ficed to produce a higher compression ratio. By utilizing £T , t '^T'T^^ can ^ used to reduce 

lossy compression methods, many new multimedif T ° f * e "ease refer to FIG. 

applications, such as video on demand, overcoming limita- «° J?** ^ Can b * 15 a comb ™ t,on ° f a 

tions on bandwidth and becoming increasingly popular. plur f y ° f ."?*»• f ch c bl ° ck » T^, 0 / a fixed 

„ . . ,, 6 ^ w number of pixels, such as 8x8 pixels. For the N frame 131 

Scientists continue to seek better compression methods to that is to be encoded, the system searches for a similar image 

reach higher and higher compression ratios. A common m N-l reference frame 132. If a block 134 in N-l reference 

result, however, is that as a method produces a higher 45 frame 132 that is similar to a block 133 in the N fiame 131 

compression ratio, the associated complexity and cost to can be found, the block 133 can be described as block 134 

implement the method tend to increase. Now we will briefly plus a vector 135. Compared to storing a whole block data, 

explain a popular lossy method, MPEG, to give a clearer a representation of another block in a referenced frame phis 

idea on compression methods of the multimedia age. a vec , or will greatly reduce the volume of data. More 

MPEG is the abbreviation of "Moving Picture Experts so importantly, in video data, since neighboring frames have 

Group". It represents a group of international standards, high levels of similarity, sacrificing some accuracy to obtain 

such as MPEG 1 and MPEG 2. Most conventional video a very high compression ratio is very powerful. This method 

CD-ROMs, including VCDs and DVDs, adopt MPEG com- is referred to as a "lossy" compression method, 

pression techniques. In the case of a movie, the date are As shown in FIG. 1(D), frame N is encoded by reference 

typically separated as a video data stream and an audio data ss to lhe preceding frame N-l. In such case, the frame N is 

stream, and both data streams are compressed with MPEG referred to as a "P frame". The MPEG technique refers to a 

compression. p frame an j f rame and a B frame j frame refers to nQ 

Please refer to FIG. 1(A). FIG. 1(A) shows an example of other frames, the P frame refers to previous frame, and the 

multimedia data. The multimedia data 111 comprise a video B frame refers to a previous frame and a latter frame. FIG. 

data stream 112 and an audio data stream 113. To play the 60 1(E) shows a frame series of a MPEG video stream that is 

multimedia data 111 properly, time parameters of the video composed of I, P, and B frames in a predetermined sequence 

data stream 112 and the audio data stream 113 are synchro- After the intra-frame compression and the inter-frame 

mzed. Theoretically, more than two data streams may be compression, the video compressed data and the audio 

combined together to provide higher resolution, and each compressed data are divided into packets and then recom- 

data stream may be compressed separately. 65 bined together in a predetermined sequence to form an 

In the MPEG video compression technique, an intra- MPEG data stream as shown in FIG. 1(F). Because dynamic 

frame compression and an inter-frame compression are both length encoding is utilized in MPEG encoding, I, P, and B 



US 6,559. 

3 

frames may be constituted by different numbers of packets. 
Playback or editing applications read the packets to acquire 
necessary data for further processing. Each packet has a 
packet header to record necessary information of the packet. 
An example of the information stored in the packet header 5 
is the time information. The time information in the packet 
header can be used to synchronize different channels, such 
as the video and sound channels. Therefore, a multimedia 
effect is presented after processing using an MPEG tech- 
nique. Further information may be found by reference to the 10 
ISO/IEC 11172 and ISO/IEC 13818 standards. 

To reduce the huge amount of data in multimedia 
applications, adopting rather complicated compression 
methods such as the MPEG method described above may 
resolve problems of storage and transmission of data. 15 
Because such compression methods are complicated, 
however, the process of compressing data takes a relatively 
long time. Furthermore, when the data is needed for play- 
back or editing, a decompression must be performed on the 
compressed data before hand. Processing time becomes a 20 
more important factor when considering large multimedia 
data such as video streams. 

Furthermore, for lossy compression methods that bring 
higher compression ratios, as each time the compression and 
decompression process is performed, the accuracy of data is 25 
undermined, a phenomenon known as "error propagation". 
In the compression methods of MPEG video streams, for 
example, because intra-frame compression or inter-frame 
compression are both lossy compression, after several times 
of decompression and recompression on a video stream, the 30 
quality of the video stream may deteriorate. Therefore, error 
propagation is a serious side-effect of lossy compression. 

If each manipulation of stored data requires compression 
and decompression prior to use or edit, much processing 35 
time may be wasted. Even worse, if the data is compressed 
with a lossy compression technique, then the quality of the 
data may degrade quickly with multiple uses. Moreover, the 
quality of the entire data file may become degraded even if 
only a portion is used or edited. 

SUMMARY OF THE INVENTION 

It is therefore one aspect of the present invention to 
provide a system and method for efficiently processing 
compressed data to reduce data inaccuracy. 45 

The first embodiment of the present invention includes a 
dispatching unit, a first processing unit, a second processing 
unit, and an integrating unit. The dispatching unit is used to 
separate a source compressed data stream into a first group 
of source segments and a second group of source segments. 50 
The first group of source segments are only necessary to 
selectively modify a status, like timestamp, by the first 
processing unit to generate a first group of target segments. 
For the first group of segments, operations of decompression 
and recompression are not generally taken. 55 

A second processing unit includes a decompressing unit, 
an operating unit, and a recompressing unit. The first group 
of source segments are firstly decompressed by the decom- 
pressing unit to generate a second group of uncompressed 
source segments. As the second group of source segments 60 
second group are decompressed, the operating unit may 
change the content. Exemplary change includes adding data, 
directing an operation to follow a rule, or mixing a param- 
eter data segment decompressed from other compressed data 
streams. After the operation of the operating unit, a second 65 
group of uncompressed target segments are obtained. The 
second group of uncompressed target segments are then 
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re compressed by the recompressing unit to generate a sec- 
ond group of target segments. 

The first group of target segments and the second group 
of target segments are then combined by the integrating unit 
to generate a target compressed data stream. 

The system is able to save time significantly because 
unnecessary decompression and recompression operations 
are skipped. Furthermore, error propagation is reduced at the 
same time. 

The present invention is also applicable to operate inter- 
frame compression in MPEG video streams. For frames that 
are not used, decompression and recompression operations 
are not taken. Therefore, efficiency can be improved, and 
unnecessary accuracy loss can be avoided. 

BRIEF DESCRIPTION OF THE DRAWINGS 

The features and advantages of the present invention are 
hereinafter described in the following detailed description of 
illustrative embodiments to be read in conjunction with the 
accompanying drawing figures, wherein like reference 
numerals are used to identify the same or similar parts in the 
similar views, and: 

FIG. 1(A) depicts a diagram showing a piece of multi- 
media data; 

FIG. 1(B) depicts a flowchart explaining an MPEG intra- 
frame compression; 

FIG. 1(C) depicts a diagram showing a mapping with 
zig-zag direction; 

FIG. 1(D) depicts a reference diagram of an MPEG 
inter-frame compression; 

FIG. 1(E) depicts a series of frames in a MPEG video 
stream; 

FIG. 1(F) depicts a MPEG stream as a series of packets; 

FIG. 2(A) depicts a diagram showing the system of the 
first embodiment; 

FIG. 2(B) depicts a flowchart showing the method of the 
first embodiment; 

FIG. 3(A) depicts a diagram showing basic hardware 
environment; 

FIG. 3(B) depicts a diagram showing a basic software 
environment; 

FIG. 4(A) depicts a diagram showing the interaction 
between a user and the application; 

FIG. 4(B) depicts a diagram showing an exemplary user 
interface; 

FIG. 5(A) depicts a diagram showing an example of the 
first basic operation; 

FIG. 5(B) depicts a diagram showing an example of the 
second basic operation; 

FIG. 5(C) depicts a diagram showing an example of the 
third basic operation; 

FIG. 6(A) depicts a diagram showing an exemplary edit 
operation; 

FIG. 6(B) depicts a diagram showing another exemplary 
edit operation; 

FIG. 6(C) depicts a diagram showing another exemplary 
edit operation; 

FIG. 6(D) depicts a diagram showing another exemplary 
edit operation; 

FIG. 7 depicts a flowchart showing an edit operation 
processing; and 

FIG. 8 depicts a diagram showing an MPEG video data 
stream. 
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DETAILED DESCRIPTION OF EXEMPLARY 
EMBODIMENTS 




Exemplary embodiments of the present invention are 06556(1 b * av0ldlD & the unneoessar y operations of compres- 
described by functional block elements and by processing slon and decompression that would otherwise result in 
steps of a flowchart. Each component shown in the block 5 unnecessar y error propagation. 

diagrams can be implemented with any number of hardware ^ second processing unit 23 suitably includes a decom- 

and/or software elements. For example, the present inven- Passing unit 231, an operating unit 232, and recompressing 
tion can use various integrated circuit elements, very large unit 233 First of ^ ^ second gr° u P of source segments 
scale integrated circuit (VLSI) components such as memory, 271 ™ c decompressed by the decompressing unit 231 to 
processors, logic circuits, look-up tables, etc. Similarly, 10 generate a second group of decompressed source segments 
software elements in the present invention may be imple- 272 - Next > ^ operating unit 232 performs an operation to 
mented with any kind of programming or scripting lan- the second group of decompressed source segments 272 to 
guages such as C, C++, JAVA, COBOL, BASIC, assembly, generate a second group of uncompressed target segments 
PERL, etc. The software elements may also include various 273 - Next > the recompressing unit 233 recompresses the 
algorithms performed by any number of data structures, 15 second group of uncompressed target segments 273 to a 
objects, processes, routines, or other programming elements. second group of target segments 274. 
Furthermore, it is noticed that the present invention may use An embodiment of the operating unit 232 may add a piece 
any conventional techniques for data transmission, signal °f data to the second group of decompressed source seg- 
delivery, data process, network controlling, etc. ments 272 or change contents of the second group of 

Please refer to FIG. 2(A) showing a first embodiment of 20 decompressed source segments 272 according to a rule. For 
the present invention. A system 20 for processing a source- example, m an MPEG compressed video data stream, the 
compressed data stream 25 suitably includes a dispatching second & on V of decompressed source segments 272 may be 
unit 21, a first processing unit 22, a second processing unit data of decompressed frames. Therefore, the operating unit 
23, and an integrating unit 24. The compressed data stream „ 232 ma y add one or more special effects, such as a color 
25 is separated into a first group of source segments 261 and chan S e or a Partial magnification or reduction into frames 
a second group of source segments 271. The source com- according to a rule. Furthermore, the operating unit 232 may 
pressed data stream 25 is a compressed ordered data set, also mix a piece of data into the frames, such as adding of 
such as an MPEG compressed image data stream, MPEG captions, logos, etc. 

compressed audio data stream, or the like. A packet 30 The embodiment of the operating unit 2323 may also 
sequence in FIG. 1(F) can be regarded as an MPEG com- operate the second group of decompressed source segments 
pressed data stream, including several MPEG compressed 272 by referencing to a parameter data segment. The param- 
image data streams and MPEG compressed audio data eler data segments may be obtained by decompressing from 
streams, for example. another compressed data stream. For example, in MPEG 

Please refer to FIG. 2(A). The dispatching unit 21 sepa- 35 video data stream > lhe fade in/out or metamorphosing effects 
rates the source compressed data stream 25 into the first may generated by making operations on one segment of 
group of source segments 261 and second group of source frames, and the operations are made by referencing to 
segments 271. Next, the dispatching unit 21 delivers the first another segment of frames. 

group of source segments 261 and the second group of Please refer to FIG. 2(A) again. After the first processing 
source segments 271 to the first processing unit 22 and the 40 umt 22 and the second processing unit 23 generate the first 
second processing unit 23, respectively. The first group of group of target segments 262 and the second group of target 
source segments 261 are portions of source compressed data segments 274 respectively, the integrating unit 24 combines 
stream 25, and these portions are not necessary to take a me first group of target segments 262 and the second group 
decompression operation before an assigned manipulation. °f target segments 274 to generate a compressed target data 
The second group of source segments 271 are the other 45 stream 28. 

portions of source compressed data stream 25, and these To summarize an example for processing on a MPEG 
portions are necessary to take a decompression operation video stream, the MPEG video stream is first separated into 
before an assigned manipulation. Besides, after processing two groups of segments. The status of the first group of 
of the first processing unit 22 and the second processing unit segments is selectively modified while the second group of 
23, the integrating unit 24 combines both results to generate 50 segments is decompressed, operated, and recompressed. 
a target compressed data stream 28. Finally, the first group of segments and the second group of 

As mentioned above, the first processing unit 22 handles segments are integrated back together to generate a new 
the first group of source segments 261 that are not necessary MPEG video stream. 

to be decompressed during processing. What the first pro- FIG. 2(B) explains the concept of the present invention in 
cessing unit 22 does is to selectively modify a status of the 55 a flowchart manner. For processing a compressed data 
first group of source segments 261. The example of the stream, the first step is to input the source compressed data 
status is the timestamp used for synchronization. After stream (step 291). The next step is to distinguish the first 
modifying the status, the first processing unit 22 generates a group of source segments from the second group of source 
first group of target segments 262 as shown in FIG. 2(A). In segments (step 292). The next step is to separate the first 
an MPEG video stream example, the first group of source 60 group of segments from the second group of segments (step 
segments 261 may be those frames that only change their 293). 

appearance time in the video. In this case, only the corre- For the first group of source segments, the status of the 
sponding timestamp in packet headers for corresponding segments is selectively modified to generate a first group of 
frames needs to be modified; the associated packets do not target segments (step 294). The status may include the 
need to be decompressed. For multimedia compressed data 65 parameter data, such as timestamp. For the second group of 
streams, the compression method used suitably avoids source segments, a series of successive steps are performed 
unnecessary operations of compression and decompression, (step 295). These steps include decompressing the second 
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group of source segments to generate a second group of 
decompressed source segments (step 2951), performing an 
operation on the second group of decompressed source 
segments to generate a second group of uncompressed target 
segments (step 2952), and recompressing the second group 5 
of uncompressed target segments to generate a second group 
of target segments (step 2953). The operation in step 2952 
includes changing the second group of decompressed source 
segments according to a rule, referring to a parameter data 
segment, and mixing the decompressed segments with a 
piece of data, etc. 1 

Next, when the first group of target segments and the 
second group of target segments are obtained, these two 
groups of target segments are combined (step 296) to 
generate a target compressed data stream (step 297). 

The typical hardware environment of the first embodi- 15 
ment is shown in FIG. 3(A). It shows a standard computer 
hardware configuration. It has a central processing unit 310, 
such as a microprocessor, and other units utilizing a system 
bus 312 to connect to each other. Additionally, the hardware 
also typically has random-access memory (RAM) 314, 20 
read-only memory (ROM) 316, input/output adapter 318 
linking the peripheral equipment, such as disk storage 
device 320, to the bus 312, user-interface adapter 322 
linking other user-interface devices, such as keyboard 324, 
mouse 326, speaker 328, microphone 332, and/or touch 25 
screen (not shown), to the bus 312. Optional communica- 
tions adapter 334 further links the computer to an optional 
communication network (such as data processing network), 
and monitor adapter 336 links the bus 312 to display devices 
338. The hardware is usually installed with an operating 3Q 
system (OS), such as Microsoft Windows operating system, 
IBM OS/2 operating system, MAC OS, UNIX, Linux oper- 
ating system, etc. The host platform may be any sort of 
personal computer or workstation, or may be implemented 
as a notebook computer, mobile communication device 35 
(such as a mobile phone), personal digital assistant (PDA), 
or other platform. 

The typical software environment of the first embodiment 
is shown in FIG. 3(B). The bottom layer is of drivers 341 for 
various kinds of hardware, the middle layer is an operating 40 
system 342, and the top layer is the multimedia applications 
343. Users use the multimedia applications 343 of the top 
layer to perform interaction when manipulating the multi- 
media data, and the multimedia applications 343 is used to 
call the operating system 342 to provide corresponding file 45 
operation, data stream operation, video outputting, etc. The 
operating system 342 communicates with hardware via the 
drivers 341 to complete jobs from the applications. 

Various embodiments of the present system, include the 
above-mentioned elements, are capable of being embedded 50 
in the multimedia applications 343, the operating system 
342, and the driver 341, to cooperate with the above men- 
tioned hardware to achieve the objective of the invention. 
Alternately, the present invention can also be partially 
implemented in the layer of drivers 341, and partially 55 
implemented in the external hardware, such as video accel- 
erator board. The present invention also can be written in 
codes to form libraries, modules, objects, daemons, or 
integrated into packages. The present invention can be 
stored in readable media of computers, such as hard disks, 60 
floppy disks, CD-ROMs, various kinds of memory and 
loaded into above mentioned software and hardware envi- 
ronment to perform the desired operation. It is also feasible 
that the system of present invention can be implemented into 
logic circuit as a part of the hardware implementation. 55 

For describing the operations of the embodiment clearly, 
in the following, MPEG data editor is taken as an example. 
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Please refer to FIG. 4(A). A user 411 communicates with 
above mentioned hardware and software environment, such 
as a computer 414, through input devices, such as an input 
device 412, 413, to manipulate MPEG data. It is to be noted 
that the user also can perform the mentioned operations 
through network or in a batch manner. 

Please refer to FIG. 4(B) for an example of a user 
operating the multimedia data through a monitor. The user 
assigns and browses data streams 421, 422 to process a 
result of a target data stream 423. The user further assigns 
processed segments 424, 425 in the source data streams, 
and/or assigns other operations 426, 427. The interface may 
also allow the user to edit MPEG data interactively. 

Manipulation on MPEG video streams may include 
operations of transition, title, and cut & merge, for example. 
FIG. 5(A), FIG. 5(B), and FIG. 5(C) describe these opera- 
tions respectively. 

Please refer to the transition operation of FIG. 5(A). The 
example shows that segments of frames of the source data 
streams SI, S2 are overlapped to generate a target data 
stream T. Frames a4 to a6 of 51 are overlapped with frames 
bl to b3 of S2. In particular, the fade in/fade out operations 
are performed on the overlapped portion. The fade in/fade 
out process involves successively reducing frame contents 
of SI, and, at the same time, successively increasing frame 
contents of S2, and recalculating the overlapped frames to 
achieve an effect of "transition". The frames al to a3 of SI 
and frames b4 to b6 of S2 are only modified in parameters 
of times tamp. In this example, the source data streams a4 to 
a6, bl to b3 need decompression and belong to second group 
of source segments mentioned above, and al to a3, b4 to b6 
belong to the first group of source segments. 

Please refer to the "title" operation shown in FIG. 5(B). 
Contents of frames a4-a6 of the source data stream SI are 
modified to generate a target data stream T. Since decom- 
pression is unnecessary, al to a3, a7 to a9 of the source data 
stream SI belong to first group of source segments. In 
contrast, a4 to a6 of SI need decompression first to generate 
corresponding new frame data a4* to a6', so a4 to a6 belong 
to the second group of source segments. 

Please refer to FIG. 5(C) for "Cut and Merge". Frames of 
a4 to a5, a6 to a9 of the source data stream SI and b7 to b9 
of the source data stream S2 are cut and merged to become 
the target data stream T with a predetermined order. In this 
example, a4 to a5, a6 to a9, and b7 to b9 do not need 
decompression and they only need to be modified in times- 
tamp. Therefore, a4 to a5, a7 to a9, b7 to b9 belong to the 
first group of source segments. 

Next, FIG. 5 and FIG. 6 are used to describe how users 
operate the compressed data stream and how these three 
basic operations mentioned above are combined to form 
manipulation. Besides, a flowchart in FIG. 7 shows the steps 
regarding to operation of FIG. 6. 

Please refer to FIG. 6(A). The compressed data stream SI 
is composed of three segments PI, P2, and P3. Now segment 
P2 is erased to recombine PI and P3. The editing in FIG. 
6(A) can be realized by several "cut & merge" operations 
mentioned above. For brevity purpose, here we use common 
object oriented language to describe the process. T is a target 
data stream object, 'Add' is a method of T, the first param- 
eter of 'Add' is a source segment, the second parameter is a 
timestamp, and timestamp denotes a starting reference time 
at which the source segment is added. Therefore, the opera- 
tions in FIG. 6(A) can be converted into basic operation 
combination of 'TAdd(Sl.P3, tl)', 'T.Add(Sl.Pl, t2)'. As 
mentioned above, this example only needs to change times- 



